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associated with the decoded timing portion, and recoding the 
timing portion using the duration parameter and a local 
reference time parameter. 
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TIMING CORRECTION METHOD AND stream. Another drawback is that the operation of the system 

APPARATUS can lead to an irregularly spaced PCR packet at the transition 

of the switching process (i.e., not ATSC compliant). 

This application is related to U.S. patent application Ser, Moreover, merely retiming the PTS and DTS is insufficient 

No. 08/864,325, filed on the same date as the present 5 to insure proper buffer conditions at a far end subscriber 

application. decoder. 

This invention was made with U.S. government support Therefore, a need exists in the art for a retiming method 

under contract number. 70 NANB5H1174. The U.S. Gov- and apparatus which allows for the insertion of opportunistic 

ernment has certain rights in this invention. data while maintaining timing compliance. It is also desir- 

The invention relates to communication systems in io able to provide a retiming method and apparatus which 

general, and more particularly, the invention relates to a insures proper buffer conditions at the far-end subscriber 

method and apparatus for retiming packetized information decoder, 
including timing information. 

BACKGROUND OF THE DISCLOSURE 15 

The disadvantages heretofore associated with the prior art 

Synchronization of a decoding and presentation process are overcome by the present invention of a retiming appa- 

for received bitstreams is a particularly important aspect of ralus anc j me thod which receives an information streams, 

real-time digital data delivery systems such as digital tele- such as variable length packetized elementary streams, 

vision systems. For example, the Moving Pictures Experts 2Q decodes the timing portions of the received streams using a 

Group (MPEG) has promulgated several standards relating i oca i tim i ng reference, calculates new timing information 

to digital data delivery systems. The first, known as MPEG-1 us i ng me decoded timing information and timing informa- 

refers to ISO/1EC standards 11172, incorporated herein by t i on related to the local timing reference, and retimes the 

reference. The second, known as MPEG-2, refers to ISO/ streams using the calculated timing information. The 

IEC standards 13818, incorporated herein by reference. A 2g retimed streams are re-encoded to form a stream which, 

compressed digital video system is described in the wne n decoded by a decoder, will reproduce the desired 

Advanced Television Systems Committee (ATSC) digital information without errors induced by timing discontinui- 

television standard document A/53, incorporated herein by ti es> 

reference. Because received data is expected to be processed n „ 4 . „• . . r 

, 4 /t . , ..it-. 1 More specifically, the invention receives an information 

at a particular rate (to match the rate at which it is generated t v • • . 1+ c 4 > . , 

*\ .„ .V ; , , ... . , * ' 3D stream comprising a plurality of timing portions and asso- 

and transmitted), loss of synchronization leads to either ■ + A n , «l *■ • *• i_ ■ 

. „, „ 7 - « ..lji i ciated paylo ad portions, the timing portions being necessary 

buffer overflow or underflow at the decoder, and as a f j a • * j - * j 1 1 j _* • * j 

~ . , . for decoduig said associated paylo ad portions to produce a 

consequence, loss or presentation and/or displaced synchro- * j j ™ . 4 . , , . , - - 

nization program at a decoder. The invention decodes each timing 

portion of the information stream, determines a duration 

There are many instances where there is a need to switch 35 parameter of a payload portion associated with the decoded 

from one encoded or compressed bitstream to another. When timing portion and recodes the timing portion of said infor- 

switching between bitstreams it is important to accurately mation stream using the duration parameter and a local 

preserve the timing information. For example, a TV studio reference time parameter. 

can switch from an ATSC bitstream stored on a server to 

another server-stored bitstream. Studio switching can also 40 BRIEF DESCRIPTION OF THE DRAWINGS 
occur between an active encoder (live camera) and a server 

(e.g., inserting commercials into a program stream), or ^ teachin g s °f me present invention can be readily 

between a studio program stream and remote feed. Home understood by considering the following detailed descrip- 

receivers can switch to a program bitstream to a bitstream tion in conjunction with the accompanying drawings, in 

from a tape player/recorder. Bitstream generators, used for 45 wmcn: 

compliance or other tests, usually loop a bitstream endlessly. FIG. 1 shows a block diagram of a transport layer 

This is a form of splicing, i.e., joining of the end of the switching and retiming system according to the invention; 

bitstream to the beginning of the bit-stream. FIG. 2 shows a block diagram of a packetized elementary 

When switching from one compressed ATSC video bit- stream layer switching and retiming system according to the 

stream to another, appropriate measures must be taken in the 50 invention; 

transmission order of the picture bitstream to assure proper FIG. 3 shows a block diagram of a video PTS-DTS 

subsequent presentation of the decoded pictures, without retiming system suitable for use in retiming systems of FIG. 

time gaps. To provide a smooth transition of program video 1 an d FIG 2* 

and audio decoding and presentation, the presentation time FIG. 4 shows a block diagram of a receiver/decoder 

stamp (PMS) and the decode time stamp (DTS) must be 55 system . 

retimed if the transport stream included PES streams. « , t , ,. 

Prior art retiming systems include a 27 MHz (local) ^ 5 sh ° WS t block ^ m of a h °™ receiver/decoder 

station clock which is utilized by a local PCR and PCRB SySt6m ' SUCh , 85 the T ? t™" " r' 0 ' *' "* 

™ , A , j * . . . . j j L improvements according to the invention; 

generator. A multiplexed transport stream is received and the *L & 

PTS, DTS and PCR timing information is detected and 60 FIG ' 6 shows a block dia g ram of a bitstream generator 

replaced by locally generated PTS, DTS and PCR timing according to the invention; and 

information to produce a retimed transport stream. The FIG. 7 shows a block diagram of an audio PTS retiming 

transport streams comprising the multiplexed transport system suitable for use in retiming systems of FIG. 1 and 

stream are constant bitrate data streams and the resultant FIG. 2. 

transport stream is likewise a constant bitrate data stream. 65 To facilitate understanding, identical or similar reference 

Thus, mere is no occasion to insert opportunistic data since numerals have been used, where possible, to designate 

any additional data will necessarily change the bitrate of the identical or similar elements that are common to the figures. 
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DETAILED DESCRIPTION 

FIG. 1 shows a block diagram of a transport layer 
switching and retiming system 100 according to the inven- 
tion. The system 100 is illustrated using a transport layer 
which is compliant with the ATSC digital television system 
and is useful in, e.g., a television studio or station environ- 
ment. A live video and audio packetized elementary stream 
(PES) encoder 110 (e.g., a camera or remote audio/video 
feed) produces a packetized elementary stream SI which is 
coupled to a transport stream encoder 115. Transport stream 
encoder 115 converts PES SI into a first transport stream 
S2T which is coupled to a transport stream switcher 130. 
Transport stream switcher 130 receives a second transport 
stream S3T from a transport stream server 120 (e.g., a video 
disk). In response to a control signal (not shown), the 
transport stream switcher 130 selects one of the two trans- 
port streams S2T, S3T, and sends it as stream S4T to a 
transport stream decoder 135. It should be noted that the 
transport stream switcher 130 may include more than two 
input streams and that the input streams may come from a 
variety of sources, e.g., bitstream servers, active encoders 
(live camera) or remote feeds, bitstream generators or the 
like. As such, the input streams are not likely to share 
information such as program clock reference (PCR), pre- 
sentation time stamp (PTS) and decode time stamp (DTS) 
timing information. 

The transport stream decoder 135 decodes the selected 
S4T transport stream to produce a packetized elementary 
stream S4P. A PES demultiplexer 137 demultiplexes stream 
S4P, if necessary, to produce audio PES S4PA and video PES 
S4PV. The audio S4PA and video S4PV streams are coupled 
to, respectively, an audio PTS retiming unit 300A and a 
video PTS -DTS retiming unit 300V. Video retiming unit 
300V decodes and retimes the old presentation time stamps 
(PTS) and decode time stamps (DTS) of the video stream 
using new timing information derived from a Program Clock 
Reference Base (PCRB) signal S9. Audio retiming unit 
300A retimes the old PTS of the audio stream using new 
timing information derived from the PCRB signal S9. 

Transport stream encoder (TSE) 150 receives the retimed 
audio S7PA and video S7PV PES streams and multiplexes 
these streams to produce an output transport stream Sll. 
TSE 150 includes a PCR generator and re timer which 
receives a 27 MHz station clock signal S10 and creates PCR 
packets that are also multiplexed into the output transport 
stream Sll. The PCR generator also produces a synchro- 
nized PCR base reference signal S9 at a 90 Khz rate (the 
station clock of 27 MHz divided by 300). The PCRB 
reference signal S9 is fed back to the retiming units 300A, 
300V for use in the PTS and DTS stamping process. 

In the retiming system 100 of FIG. 1, the selected trans- 
port stream S4T is decoded to the PES layer so that the PTS 
and DTS of the included PES streams can be over written 
with net relative time extracted from a delayed version of the 
PCRB. Decoding to the PES layer insures that the relation- 
ships between visual and aural presentations (i.e., "lip 
sync") are strictly adhered to. In this manner, a major aspect 
of seamless splicing is accomplished; namely, processing 
the PTS and DTS in the PES layer and the PCR in the 
Transport layer to accurately define these signals in the 
resultant bitstream Sll. 

The retiming system 100 of FIG. 1 also allows for data 
flow control to insure proper buffer management in the 
subscriber PES Decoder. By algorithmic methods (which are 
discussed later with respect to FIG. 3) the subscriber-side 
buffer occupancy can be determined and, if the buffer is too 
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full, the retiming units 300A, 300V can reduce their bitrates 
(or delay delivery) and retime the audio S7PA and video 
S7PV PES streams to provide time for the subscriber-side 
buffers to unload. Additionally, the transport stream encoder 

5 150 can issue a RECEIVER NOT READY (S8) signal to the 
retiming units 300A, 300V to halt delivery of streams S7PA 
and S7PV if the TSE buffers are full. Moreover, the retiming 
units 300A, 300V may generate a buffer control signal S12 
which can be used to regulate the data production in the 
video/audio encoders (for live bitstreams) or the servers (for 
stored bitstreams). The buffer control signal S12 is depicted 
in FIG. 1 as being generated by the video PTS -DTS retiming 
unit 300V, though the signal may also be generated in, e.g., 
the audio PTS retiming unit 300A. 

15 The demultiplexing of PES S4P and subsequent remulti- 
plexing of the elementary streams S7PA and S7PV per the 
system 100 of FIG, 1 advantageously allows for either 
variable bitrate or a constant bitrate streams. In the former 
case, a data stream S5 (if available) can be extracted from 

20 the selected transport stream S4T by the transport stream 
decoder 135. The extracted data stream S5 may be, e.g., a 
private or subscriber data stream which is being included in 
the transport stream as an add-on service to the far-end 
subscriber. The extracted data stream S5 may also carry 

25 information useful to the television studio or station. The 
extracted data stream S5 may be manipulated by a data 
processing arrangement (not shown) and coupled to TSE 
150 as part of an opportunistic data stream S6. The oppor- 
tunistic data stream may also include other data streams 

30 ( e S > commercials, station identification, pay per view or 
subscriber verification codes, etc.). The opportunistic data 
stream S6 is encoded by TSE 150 and included in the output 
transport stream Sll. 
The demultiplexing of PES S4P and subsequent remulti- 

35 plexing of the elementary streams S7PA and S7PV per the 
system 100 of FIG. 1 addresses another problem in the prior 
art arrangement; namely, the requirement of locking the byte 
clocks of the presently selected stream and the stream to be 
selected next to a single 27 MHz clock source. By decoding 

40 the incoming transport streams to the PES layer, the differ- 
ential errors (i.e., tolerance limit to tolerance limit) are 
stripped away and the resultant transport stream Sll from 
TSE 150 is compliant with the timing requirements of the 
ATSC system. Moreover, by controlling the transport stream 

45 PCR stamping process, the ATSC requirement of sending a 
PCR reference packet every 100 mS may be consistently 
adhered to. 

FIG. 2 shows a block diagram of a preferred PES layer 
switching and retiming system 200 according to the inven- 
50 tion. The system 200 of FIG. 2 is illustrated using a PES 
layer which is compliant with the ATSC digital television 
system and is useful in, e.g., a television studio or station 
environment. 

In the system 200 of FIG. 2 a live video and audio PES 
55 encoder 210 (e.g., a camera or remote audio/video feed) 
produces audio S2PA and video S2PV packetized elemen- 
tary streams which are coupled to a PES switcher 230. PES 
switcher 230 also receives a second pair of audio S3PA and 
video S3PV packetized elementary streams from a PES 
60 server 220 (e.g., a video disk). In response to a control signal 
(not shown), the PES switcher 230 selects and couples one 
pair of audio S4PA and video S4PV packetized elementary 
streams to respective audio 300A and video 300V retiming 
units. The video PTS-DTS retiming unit 300V and audio 
65 PTS retiming unit 300A perform in substantially the same 
manner as previously described with respect to FIG. 1 and 
will be discussed in detail with respect to FIGS. 3 and 7. TSE 
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250 operates in substantially the same manner as previously 
described with respect to the TSE 150 in the system 100 of 
FIG. 1 and will not be discussed further. 

The process for PTS and DTS retiming, and the interac- 
tions of this process with the TSE (i.e., for PCR insertion, 5 
PCRB generation, and data flow control using the stop 
feature) and the PES Encoders (for data flow control using 
the inherently contained buffers) are important aspects of the 
invention that will be discussed next. 

FIG. 3 shows a block diagram of a video PTS-DTS 10 
retiming system 300 suitable for use in the systems of FIGS. 
1 and 2. An audio PTS retiming unit is discussed below with 
respect to FIG. 7. In general, the following description of the 
video retiming unit 300V is also applicable to similarly 
labeled portions of the audio retiming unit 300A. The 15 
PT-DTS retiming unit 300V receives a PES input bitstream 
S 4P from a PES switcher 230 (PES layer retiming system 
200) or a PES demultiplexer 137 (transport layer retiming 
system 100) and delivers a PES output bitstream S7P to a 
TSE 150 in the studio or station case (other cases, such as 2Q 
the home recorder, will be covered subsequently). The TSE 
performs the PCR stamping at its output. The TSE feeds 
back a synchronized PCR base reference at a 90 KHz rate 
(the station clock of 27 MFHz divided by 300). The PTS- 
DTS retiming unit operates as follows. 25 

PCRB serial to parallel converter 312 converts the PCRB 
signal S9 from TSE 150 into a parallel (byte) stream S301 
which is coupled to an adder 314 and a subtracter 322. Adder 
314 adds the parallel PCRB S301 to a decoder delay time 
S304 to produce an adjusted PCRB time reference S302 30 
which is stored in an adjusted time reference register 316. 
The adjusted PCRB time reference S302 reflects the value of 
the PCRB count when a first picture may be removed from 
a video buffer and decoded at the far-end decoder. 

The addition of a decoder delay time S304 provides a 35 
mechanism for achieving a particular video buffer fullness 
level in a decoder of the type which uses the DTS for buffer 
control. The additional delay time causes the decoder to 
maintain a desired level of buffer utilization, e.g., fill video 
buffer with a certain number of pictures (perhaps all of the 40 
pictures representing the end-to-end-delay). One or more of 
the following factors may be used to determine the decoder 
delay time S304; the header entries of the VBV buffer size, 
the VBV delay and frame rate, the number of bytes in a 
frame and the number of frames (i.e., a frame byte count) in 45 
the decoder buffer. 

The method for using the delay time S304 to control the 
fullness of the decoder buffer is as follows. The adjusted 
PCRB is used as the DTS entry, as will be described shortly. 
When a video PES is present at the output of the video buffer 50 
at the decoder, the DTS (Decode Time Stamp) contained in 
the PES header is compared to the PCRB real-time clock 
reference derived from the PCR (Program Clock Reference) 
packets in the transport stream. The PES must await align- 
ment of the DTS with the recovered PCRB. This alignment 55 
takes place after the period of time that the value of the delay 
time S304 indicates, because the delay time S304 was added 
to the PCRB at the encoder, and this adjusted PCRB is used 
as the DTS entry. 

At a turn-on initialization time, an initialization command 60 
S306 causes a selector 318 to transfer the stored time 
reference S303 into a new time stamp register 320. Each 
time a PES header is decoded in the input bit stream S4P, an 
offset number, stored in the frame rate (FR) register 308, is 
added to the new time stamp S308 by an adder 310. The 65 
result of this addition is returned to the new time stamp 
register 320 via selector 318. 
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The adder 310 offset number is specifically the number of 
cycles of 90 KHz contained in one frame time of the 
prevailing frame rate. The adder 310 offset numbers are 
shown in Table 1 (third column), and are automatically set 
as follows. A PES, sequence, pix and VBV delay decoder 
302 detects a sequence start code in the input data stream 
S4P and produces a sequence detect signal S312. A frame 
rate decoder 304 extracts the four-bit Frame Rate Code S310 
(first column of Table 1). The binary value of the frame rate 
code S310 corresponds to the frame rates listed in the second 
column of Table 1, as per the ATSC specification. A Look- 
Up-Table (LUT) 306 provides the offset number S311 (third 
column of Table 1) to frame rate register 308. In response to 
sequence detect signal S312, frame rate register 308 pro- 
vides the stored offset number to adder 310. It should be 
noted that since PES streams can have a plurality of 
sequences of intermixed frame rates, the PTS-DTS retiming 
unit of the invention automatically calculates all of the time 
stamp updates. 

TABLE 1 



FRAME RATE CODE 



IN 

SEQUENCE HEADER 


FRAME RATE 
VALUE (Hz) 


ADDER 310 OFFSET 
NUMBER 


0001 


24/1.001 (23.967...) 


3753.75 


0010 


24 


3750 


0011 


25 


3600 


0100 


30/3.001 (29.97...) 


3003 


0101 


30 


3000 


0110 


50 


1800 


0111 


60/1.001(59.94...) 


1501.5 


1000 


60 


1500 



The next step is to calculate the new PTS and DTS. DTS 
decode unit 328 decodes the DTS, if present, in the input 
data stream S4P to produce a DTS signal S321. DTS decode 
unit 328 also provides an indication flag S323 which is high 
(binary 1) if the input data stream S4P includes a DTS, and 
low (binary 0) if there is only a PTS (i.e., B-Frame 
operation). The DTS indication signal S323 is coupled to 
AND unit 334 and DTS update unit 340. The new DTS S327 
is taken from the new time stamp register 320 only if the 
DTS flag S323 indicates the presence of a DTS in the input 
data stream S4P. 

PTS decode unit 326 decodes the PTS in the input data 
stream S4P to produce a PTS signal S320. The difference 
S322 between the existing PTS S320 and DTS S321 is 
provided by subtractor 332 to AND unit 334. The new PT 
S326 is calculated by adding the subtractor 332 output 
number S322 to the number S308 stored in the new time 
stamp register 320. This operation takes place in adder 336. 
Notice that when there is no DTS present (i.e., DTS FlagoO), 
then the input S324 to adder 336 from AND unit 334 is zero 
(because of the operation of the AND unit 334) and new PTS 
S326 is the number S308 stored in the new time stamp 
register 320. 

The new PTS S326 and the new DTS S327 are stored in 
respective update registers 338,340. A multiplex unit 344 
inserts the new PTS S326 and DTS S327 values into the 
output data stream S7P in place of the old values. A 
controller 343 tracks the location of the appropriate bits in 
the outgoing data stream S7P, and causes the multiplex unit 
344 to insert the updates in the correct positions in the 
stream. The outgoing data stream S7P proceeds to the 
transport stream encoder 150 in the usual manner. 

The PTS-DTS retiming unit 300V also provides the 
ability to manage end-to-end delay in the system by account- 
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20 



ing for buffer behavior in a far-end decoder (e.g., a set-top 
receiver) and buffer or bitstream generation behavior in the 
bitstream sources (e.g., encoder 210 and server 220), as will 
now be described. 

PES, sequence, pix and VBV delay decoder 302 detects 5 
the start of a PES packet in the input data stream S4P and 
produces a PES detect signal S313 which is coupled to the 
new time stamp register 320. At each PES interval, the new 
time stamp register 320 is updated with a fixed constant 
S318 which is determined by the prevailing frame rate. The 1Q 
updated contents S308 of the new time stamp register 320 
are compared with the contents S303 of the adjusted time 
reference register 316 by a comparator 323. The comparator 
result (i.e., less than, greater than, or equal to) provides an 
indication of the rate at which the PES data is being 
processed (i.e., too slowly, too quickly, or just right). If the 
PES streams S4P are being issued too slowly by, e.g., the 
live video encoder 210 or server 220, the encoder 210 or 
server 220 output buffers (not shown) may be read or 
emptied at a maximum rate. If the comparison indicates that 
processing is proceeding too quickly, the reading of the 
buffers in the encoders or servers can be halted or reduced. 
A buffer control signal S12 maybe used to control encoder 
210 or server 220 buffer utilization. 

One particular apparatus that uses the buffer control signal 25 
S12 is described in commonly assigned U.S. patent appli- 
cation Ser. No. 08/864,325, filed simultaneously herewith 
(Attorney Docket 12408) and incorporated herein by refer- 
ence. In response to the flag, this apparatus adds or drops 
frames of information such that the number of bits is, 3Q 
respectively, increased or decreased. 

Additionally, there is ordinarily a buffer (not shown) at the 
input to the TSE 150. If the TSE buffer is receiving data at 
too high a rate the TSE may supply a RECEIVER NOT 
READY signal S8 to the PTS-DTS retiming unit 300. The 35 
RECEIVER NOT READY signal S8 is coupled to a logic 
unit 325 at the output of comparator 323. The logic unit 
responsively generates a buffer control signal S12 which 
halts or reduces the bitrate generation of the encoders 
210,220. 40 

The above-described buffer control method exemplifies 
an important advantage in dealing with PES switching. The 
video PES data rate can be variable, as compared to the 
constant bitrate nature of transport stream switching, where 
stopping the transport bitstream is forbidden. Thus, the 45 
variable bitrate allows insertion of opportunistic data, as 
shown in FIGS. 1 and 2, which is not possible with constant 
bitrate data. 

The PTS-DTS retiming unit of FIG. 3 has the ability to 
calculate the VBV delay number S3 16 and insert it (via 50 
multiplexer 344) into the 16 bit field within the Picture 
Header. The output of the PCRB serial to parallel converter 
312 (i.e., the current time) is subtracted from the contents of 
the new time stamp register 320 in subtracter 322. PES, 
sequence, pix and VBV delay decoder 302 detects a picture 55 
header in the input data stream S4P and produces a pix detect 
signal S314 which is coupled to the VBV delay update 
register 324. In response to the pix detect signal S314, the 
VBV delay update register 324 stores the output S3 15 of 
sub tractor 322. This stored number S3 16 represents the time, go 
in 90 KHz cycles, that the picture will reside in the far-end 
decoder buffer, namely, the VBV delay parameter. Urns, the 
invention creates a ATSC-compliant bitstream without using 
the ATSC mode in which the VBV delay fields are set to 
hexadecimal FFFF. 65 

Temporal reference in the Picture Header must also be 
adjusted to reflect the appropriate display order of the 



pictures in a group of pictures (GOP) as specified by the 
ATSC standard. When the input data stream S4P switches to 
a sequence header followed by a GOP Header, the temporal 
reference will be properly set to start at zero in the currently 
selected stream so that no adjustment is needed as the 
selected stream switches to another stream. However, if the 
GOP Header does not follow a sequence header then the new 
temporal reference value must be properly updated in the 
newly selected stream. The new temporal reference can be 
determined by monitoring the previously selected stream. 
New succeeding temporal reference values can be deter- 
mined by monitoring the original value relationships on a 
frame by frame basis and updating the new temporal refer- 
ence value with values that refer to the temporal reference 
value at the time the streams were switched. This technique 
will achieve the result of maintaining the display ordering of 
the temporal reference, even though the pictures are in 
transmission order. 

VBV buffer fullness must also be monitored to properly 
control the flow of the video stream to the decoder. At 
initialization (i.e., initialization command S306 asserted), 
the number of bytes output from the PTS-DTS retiming unit 
300V are stored on a frame by frame basis. The current time, 
decode time for each frame, the number of bytes per frame, 
and the VBV buffer size are used to determine the fullness 
of the VBV buffer. 

At initialization the VBV buffer is empty. As the first 
frames are output to the VBV buffer, a running total of the 
bytes in the VBV buffer is maintained by a frame byte 
counter 354. If this total equals the VBV buffer size, an 
output signal S3 17 from the frame byte counter 354 to the 
logic unit 325 causes the logic unit 325 to prevent data from 
being output from the PTS-DTS retiming unit 300V. 

Two modes of VBV buffer operation are possible as 
described in Annex C of the MPEG2 specification. The first 
mode (Mode A) requires VBV delay to be set to hexadeci- 
mal FFFF. The second mode (Mode B) requires the VBV 
delay to not be set to hexadecimal FFFF, but instead reflect 
the true value of VBV delay in the VBV buffer. 

During Mode A initialization, bytes are output to the VBV 
buffer until the buffer is full as indicated by frame byte 
counter 354. Once the VBV buffer is full, data is extracted 
from the VBV buffer as described in Annex C of the MPEG2 
specification. 

During Mode B initialization, bytes are output to the VBV 
buffer until the calculated value of VBV delay is greater than 
or equal to the VBV delay value in the incoming stream S4P. 
The incoming stream should be self consistent and MPEG2 
compliant in that the buffer will not over/under flow using 
the values contained in the stream. The calculated value of 
VBV buffer fullness contained in frame byte counter 354 is 
used to prevent under/over flow of the VBV buffer. Data 
flow to the VBV buffer can be regulated using this calculated 
value if the incoming stream for some reason is not consis- 
tent. The DTS - PTS values in the stream should be 
consistent with the VBV delay values contained in the 
stream. 

When switching to a stream in Mode A, the hexadecimal 
FFFF value of VBV delay in the next-selected stream is 
overwritten with the calculated value maintained in the 
PTS-DTS retiming section. When switching to a stream in 
Mode B, the hex value of VBV delay in the next-selected 
stream is overwritten, when necessary, with the calculated 
value maintained in the PTS-DTS retiming section 300V 
until the calculated VBV buffer value and the next-selected 
stream VBV delay value are aligned. Otherwise the incom- 
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ing stream data flow is held until the VBV delays align. This The audio PTS retiming unit 300A receives a PES input 

process takes advantage of the re-multiplexing operation bitstream S4P from a PES switcher 230 (PES layer retiming 

that is occurring in the Transport Stream Encoder that system 200) or a PES demultiplexer 137 (transport layer 

follows the retiming circuit. During the alignment period, retiming system 100) and delivers a PES output bitstream 

data may be output to the VBV buffer at maximum video bit 5 S7P to a TSE 150. The TSE performs the PCR stamping at 

rate, to increase the VBV delay in the VBV buffer. This its out P ut - ^ TSE feeds back a synchronized PCR base 

process can continue until the VBV delays align as long as reference at a 90 KHz rate (the station clock of 27 DMz 

the buffer does not become full. J^JJ b V 300 )' ™ e PTS " DTS retimin g umt °P erates as 

In steady state operation, a frame of data is removed from ° AtTturn-on initialization time, an initialization command 

the VBV buffer when the current time matches the decode w ^ a ^ {q ^ stored ^ 

time stamp. The number of bytes in that frame are subtracted re f e rence S303 into a new time stamp register 320. Each 

from the running total kept in the frame byte counter of the time an audio pES is decoded m me input bit stream S4P, an 

PTS-DTS retiming unit 300V If the VBV buffer had been ofifeet number , stored in the frame rate (FR) register 308A, 

full, space has now been made for more data to be sent. If ^ added t0 the new lime stamp S308Aby an adder 310. The 

all of the end-to-end delay data had been in the VBV buffer, is regult of this addition ^ returned to the new time stamp 

the frame exiting the VBV buffer at the decode time initiates register 320 via selector 318. 

the sending of the next frame. M audio pES decoder 3()2A detects ^ audio p£s s(aft 

When the VBV buffer has become full, prior to DTS of the code { n tne input data stream S4P and produces an audio PES 

first frame in the buffer minus the PCR base becoming equal de tect signal S3 12. A frame rate decoder 304 extracts a 

to the VBV delay, the above-described VBV delay process- four-bit Frame Rate Code S310. A Look-Up-Table (LUT) 

ing function operates by calculating a new VBV delay 306A provides an offset number S311 A to frame rate register 

number and overwriting the VBV delay number present in 308A. In response to audio PES detect signal S312A, frame 

the input stream S4P, thus maintaining MPEG-2 compli- rate register 308 A provides the stored offset number to adder 

ance. This method provides control of the VBV delay 310. Additionally, audio PES detect signal S312 is coupled 

number on a frame by frame basis until such time as the t0 a new t i me stamp register 320. At each PES interval, the 

VBV delay in stream S4P matches the calculated VBV delay new ume stamp register 320 is updated with a fixed constant 

in the decoder VBV buffer. A second (optional) VBV delay S318 which is determined by the prevailing frame rate, 

processing function may be implemented to provide finer p TS decode U nit 326 decodes the PTS in the input data 

control, thus allowing a smoother transition between the stream S4P to produce a PTS signal S320. The decoded PTS 

presently-selected stream and the next-selected stream. indicates that the number S308 stored in the new time stamp 

The optional second VBV delay processing function may register 320 should be coupled to a multiplex unit 244 as a 

be incorporated into the PTS-DTS retiming unit 300 as new PTS S326. A controller 343 tracks the location of the 

follows. PES, sequence, pix and VBV delay decoder 302 appropriate bits in the outgoing data stream S7P, and causes 

detects the VBV delay parameter include within the PES 35 the multiplex unit 344 to insert the updates in the correct 

picture header of input data stream S4R The detected VBV positions in the stream. The outgoing data stream S7P 

delay S329 is coupled to a second comparator 327 and proceeds to the transport stream encoder 150 in the usual 

compared to the updated VBV delay number S315 produced manner. 

by subtractor 322. The result of this comparison S330 is piG. 4 shows a block diagram of a receiver/decoder 

coupled to logic unit 325, where an additional calculation of 4Q syste m 400 which may be used in, e.g., a "far end decoder" 

decoder buffer utilization is performed. The additional cal- or home ATSC television system. The transport stream 

culation notes the size of the difference between the two decoder (TSD) 405 decodes an input transport stream (from, 

VBV delays and, in response, acts in conjunction with the e<&f a direct broadcast system) in the usual manner to 

output of comparator 323 to generate the buffer control produce a PES data stream and to also provide PCR and 

signal S12 for regulating data production in the video/audio 4S P crb data to a PES decoder 415. Decoder 415 decodes the 

encoders (for live bitstreams) or the servers (for stored audio and video PES streams in the usual manner to generate 

bitstreams). an output signal which may be coupled to an appropriate 

It must be noted that the VBV delay number present in the audio/video system (not shown) for audio and video repro- 

input stream was initially calculated by an encoder (not duction. The system of FIG. 4 does not provide for switching 

shown) to maintain the end to end delay of the system as 50 to a recorded sequence (e.g., video disk, CD, computer 

controlled by the buffer at the decoder. By measuring the storage device and the like). 

displacement of the VBV delay being produced by subtrac- FIG. 5 shows a block diagram of a home receiver/decoder 

tor 322 and the VBV delay used in the presently selected system 500, such as the system 400 shown in FIG. 4, and 

stream S4P, the logic unit 325 can modulate buffer control including improvements according to the invention. To 

signal S12 more precisely. The buffer control signal S12, as 55 accommodate switching to a recorded sequence, the system 

previously discussed, may be used to speed up or slow down 500 of FIG. 5 includes bitstream server 520, illustratively a 

the data rate of a bitstream source. home recorder tape server, which stores program material in 

FIG. 7 shows a block diagram of an audio PTS retiming a PES format. As previously discussed with respect to FIG. 

system 300A suitable for use in retiming systems of FIG. 1 2, PES storage is more efficient than transport stream storage 

and FIG. 2. The audio PTS retiming unit 300A is essentially 60 due to the avoidance of storing transport stream overhead, 

a functional subset of the video PTS-DTS retiming unit TSD 505 operates in substantially the same manner as the 

300V, described above with respect to FIG. 3, As such, only TSD 405 of FIG. 4. Moreover, PES switcher 530 and server 

the differences between the video unit 300V and the audio 520 operate in substantially the same manner as, 

unit 300A will be discussed. It should be noted that the audio respectively, PES switcher 230 and PES server 220 of FIG. 

PTS retiming system 300A does not include the VBV 65 2. Also, PTS-DTS retiming unit 300 operates in substantially 

management structure shown in the video PTS-DTS retim- same manner as PTS-DTS retiming unit 300 of FIGS. 2 and 

ing system 300V of FIG. 3. 3. 
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A buffer (not shown) to allow flow control (to maintain 
proper PES decoder buffer fullness) can be included in 
recorder/server 520 or in PES switcher 530. By locking the 
PTS-DTS retiming unit 300 to the transport decoder PCRB 
signals insures smooth switching between off-air and 5 
recorded bitstreams. In the absence of off-air signals, the 
PTS-DTS retiming unit 300 operates from a free-running 
PCR timing source, such as a voltage controlled crystal 
oscillator (VCXO) within the Transport Decoder. 

FIG. 6 shows a block diagram of a bitstream generator 10 
600 according to the invention. The bitstream generator 
includes a bitstream server 620 which stores, e.g., audio and 
video test streams in PES format. The stored PES data is 
coupled to a PTS-DTS retiming unit 300 which operates in 
substantially same manner as PTS-DTS retiming unit 300 of 15 
FIGS. 2 and 3. The retiming unit 300 produces a continu- 
ously looping audio and/or video bitstream which is coupled 
to a TSE 650. TSE 650 operates in substantially the same 
manner as TSE 250 of the system 200 of FIG. 2 to produce 
a resultant transport stream Sll. 20 

The continuously looping bitstream system of FIG. 6 
addresses a problem similar to that which is addressed by, 
e.g., the system 200 of FIG. 2, namely the seamless joining 
together of two bitstreams having unrelated timing informa- 
tion. In the previously described cases of switching between 25 
two bitstreams, the bitstreams are from different sources. In 
the bitstream generator scenario case, the "switching" is 
from the end of a bitstream to the beginning of the same 
bitstream. Thus, the timing information of the "two" (i.e., 
beginning and end) bitstreams is not synchronized and TSE 30 
650 would possibly produce a non-compliant transport 
stream without the retiming of the second (i.e., beginning) 
bitstream. 

Another application of the invention is in the use of 35 
Interactive Digital Video Disc (DVD) servers. If bitstreams 
are recorded as PES streams on Digital Video Discs, then 
interactive frame switching on the disc is possible using the 
retiming methods described. For example, in a DVD-based 
interactive game a user may select one of a number of 4Q 
possible actions (e.g., open a door, pick up a weapon, 
converse with someone or something, etc.), each of the 
actions requiring the playing of a different audio/video 
sequence. These sequences are stored on the DVD with 
predetermined timing information which may need to be 45 
retimed when switching from one stored sequence to 
another. 

The above-described embodiments of the invention illus- 
trate the method used by the invention performing various 
operations on, e.g., compressed ATSC transport and PES 50 
bitstreams while maintaining compliance with the relevant 
standards. This method regulates flow control, such that 
buffer overflow or buffer underflow may be avoided in a 
straightforward manner. The invention also provides a way 
to exit a bitstream in multiple places, not simply the pre- 55 
defined splice points described in the ATSC standard 
(referring to the MPEG standard). It must be noted that a 
stream should be entered at, e.g., an I-frame or anchor frame. 

Although various embodiments which incorporate the 
teachings of the present invention have been shown and go 
described in detail herein, those skilled in the art can readily 
devise many other varied embodiments that still incorporate 
these teachings. The invention has broad applicability. 

For example, the invention is useful in studios to perform 
splicing between different services which are not locked. 65 
The invention allows home (i.e., consumer) recorders to 
create high quality splices; avoiding waviness, blue-fields 
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and other non-studio quality artifacts. The invention allows 
switching from, e.g., a television picture to a VCR picture 
w/o disruption in image quality. 

The invention is useful for any data stream, such as 
auxiliary or control data, which may be transmitted or stored 
in any storage medium and remultiplexed back into the 
bitstream. For example, if broadcasters wish to sell excess 
capacity in a datastream to a data delivery service (e.g., 
"pointcast" information service, program guides, assorted 
telecom services and the like) then the broadcaster will need 
to easily manipulate the data streams. The invention pro- 
vides a tool to accomplish this manipulation. 

These and other embodiments are intended to be within 
the scope of the following claims. 

What is claimed is: 

1. A time base correction method comprising the steps of: 

(A) receiving an information stream, said received infor- 
mation stream comprising a compressed video stream 
having a plurality of timing portions and associated 
payload portions, said timing portions being necessary 
for decoding said associated payload portions to pro- 
duce a video program at a decoder; 

(B) decoding a timing portion including a presentation 
time stamp of said information stream; 

(C) determining a duration parameter of a payload portion 
associated with said decoded timing portion; 

(D) modifying said decoded presentation time stamp 
using said duration parameter to form a new presenta- 
tion time stamp; 

(E) recoding said timing portion including said new 
presentation time stamp of said information stream; and 

(F) repeating steps (B) through (D) for each of said 
plurality of timing portions and associated payload 
portions of said received information stream to produce 
an output information stream, said output information 
stream comprising said compressed video stream, said 
compressed video stream including modified timing 
portions. 

2. The method of claim 1 wherein: 

step (C) further comprises the step of determining a size 

parameter of said payload portion associated with said 

decoded timing portion; and 
step (D) further comprises the steps of: 

comparing said duration parameter to a decode buffer 
utilization parameter; and 

responsively reducing a rate of receiving said informa- 
tion stream if said comparison indicates that said 
decode buffer utilization exceeds a predetermined 
threshold level 

3. The method of claim 2 wherein: 
step (D) further comprises the step of: 

responsively increasing a rate of receiving said infor- 
mation stream if said comparison indicates that said 
decode buffer utilization is less than a predetermined 
threshold level. 

4. The method of claim 1 wherein; 

step (C) further comprises the step of determining a size 

parameter of said payload portion associated with said 

decoded timing portion; and 
step (D) further comprises the steps of: 

comparing said duration parameter to a decode buffer 
utilization parameter; and 

responsively reducing a rate of receiving said informa- 
tion stream if said comparison indicates that said 
decode buffer utilization exceeds a predetermined 
threshold level. 
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5. A method for retiming a bitstream, said bitstream 
comprising timing portions and respective compressed pay- 
load portions, said method comprising the steps of: 

generating a local time stamp in response to a local timing 
signal; 5 

retrieving a presentation time stamp from said timing 
portions of said bitstream; 

adding time stamp to produce a new presentation time 
stamp; and 1Q 

inserting said new presentation time stamp into said 
bitstream to produce a retimed bitstream, said retimed 
bitstream comprising retimed portions and respective 
compressed payload portions. 

6. The method of claim 5 further comprising the steps of: 15 
retrieving a decode time stamp from said timing portions 

of said bitstream; 
subtracting said retrieved decode time stamp from said 

new presentation time stamp; and 
inserting said generated time stamp into said bitstream as 20 

new decode time stamp. 

7. The method of claim 5, wherein said generating step 
comprises the steps of: 

adding a delay parameter to said local timing source 25 
signal to produce an adjusted timing signal; and 

storing said adjusted timing signal in response to a control 
signal representative of the start of an information 
packet in said bitstream. 

8. The method of claim 7, further comprising the steps of: 30 
producing a buffer control signal, said buffer control 

signal indicative of a desired flow of data in said 
bitstream; 

comparing said adjusted timing signal to said stored 
timing signal; and 35 
in the case of said adjusted timing signal exceeding said 
stored timing signal, causing said buffer control 
signal to indicate that said desired flow is reduced; 
in the case of said stored timing signal exceeding said 
adjusted timing signal, causing said buffer control 40 
signal to indicate that said desired flow is increased. 

9. The method of claim 7, further comprising the steps of: 
producing a buffer control signal, said buffer control 

signal indicative of a desired flow of data in said 
bitstream; 45 
subtracting said generated time stamp from said adjusted 

timing signal to produce a first difference signal; 
comparing said first difference signal to a buffer delay 
parameter associated with said bitstream; and 50 
in the case of said first difference signal exceeding said 
buffer delay parameter associated with said 
bitstream, causing said buffer control signal to indi- 
cate that said desired flow is increased; 
in the case of said buffer delay parameter associated 55 
with said bitstream exceeding said first difference 
signal, causing said buffer control signal to indicate 
that said desired flow is decreased. 

10. The method of claim 7, further comprising the steps 

0f: 60 
producing a buffer control signal, said buffer control 
signal indicative of a desired flow of data in said 
bitstream; 

monitoring an indicium of the number of bytes in a 
decoder buffer; and 65 
in the case of said number of bytes in said decoder 
buffer exceeding a predetermined maximum amount, 
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causing said buffer control signal to indicate that said 
desired flow is decreased; 
in the case of said number of bytes in said decoder 
buffer being below a predetermined minimum 
amount, causing said buffer control signal to indicate 
that said desired flow is increased. 

11. Apparatus suitable for retiming a bitstream in response 
to a local timing source, said bitstream comprising timing 
portions and respective encoded payload portions said tim- 
ing portion being useful in decoding respective payload 
portions, said apparatus comprising: 

a time stamp generator, responsive to a timing signal, for 
generating a time stamp; 

a first decoder for retrieving a presentation time stamp 
from said timing portions of said bitstream; 

an adder for adding said generated time stamp to said 
retrieved presentation time stamp to produce a new 
presentation time stamp; and 

a multiplexer, coupled to said adder, for inserting said new 
presentation time stamp into said bitstream to produce 
a retimed bitstream, said retimed bitstream comprising 
retimed timing portions and respective encoded pay- 
load portions. 

12. The apparatus of claim 11 further comprising: 

a second decoder for retrieving a decode time stamp from 
said timing portions of said bitstream; and 

a sub tractor for subtracting said retrieved decode time 
stamp from said new presentation time stamp; 

said multiplexer inserting said generated time stamp into 
said bitstream as a new decode time stamp. 

13. The apparatus of claim 11, wherein: 
said time stamp generator comprises: 

an adder, for adding a delay parameter to said timing 
signal, to produce an adjusted timing signal; and 

a storage element for storing said adjusted timing signal 
in response to a control signal representative of the 
start of an information packet in said bitstream. 

14. The apparatus of claim 13 further comprising: 

a logic unit for producing a buffer control signal, said 
buffer control signal indicative of a desired flow of data 
in said bitstream; and 

a first comparator, coupled to said time stamp generator 
and said logic unit, for comparing said adjusted timing 
signal to said stored adjusted timing signal and respon- 
sively producing an output signal; 

said logic unit, in response to said first comparator output 
signal indicating that said adjusted timing signal 
exceeds said stored adjusted timing signal, causing said 
buffer control signal to indicate that said desired flow is 
reduced; and 

said logic unit, in response to said first comparator output 
signal indicating that said stored adjusted timing signal 
exceeds said adjusted timing signal, causing said buffer 
control signal to indicate that said desired flow is 
increased. 

15. The apparatus of claim 13 further comprising: 

a logic unit for producing a buffer control signal, said 
buffer control signal indicative of a desired flow of data 
in said bitstream; 

a second subtractor, coupled to said timing generator and 
said timing source, for subtracting said generated time 
stamp from said adjusted timing signal to produce a 
first difference signal; and 

a second comparator, coupled to said second subtractor 
and said logic unit, for comparing said first difference 
signal to a buffer delay parameter associated with said 
bitstream; 
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said logic unit, in response to said second comparator 
output signal indicating that said first difference signal 
exceeds said buffer delay parameter associated with 
said bitstream, causing said buffer control signal to 
indicate that said desired flow is increased; 5 

said logic unit, in response to said second comparator 
output signal indicating that said buffer delay parameter 
associated with said bitstream exceeds said first differ- 
ence signal, causing said buffer control signal to indi- 
cate that said desired flow is decreased. 1C 

16. The apparatus of claim 13 further comprising: 

a logic unit for producing a buffer control signal, said 
buffer control signal indicative of a desired flow of data 
in said bitstream; and 

a frame byte counter, coupled to said logic unit, for 
counting a number of bytes stored in a decoder buffer; 

said logic unit, in response to said number of bytes stored 
in said decoder buffer exceeding a predetermined maxi- 
mum amount, causing said buffer control signal to 2 o 
indicate that said desired flow is decreased; 

said logic unit, in response to said number of bytes stored 
in said decoder buffer being below a predetermined 
minimum amount, causing said buffer control signal to 
indicate that said desired flow is increased. 25 

17. The apparatus of claim 11 wherein said output bit- 
stream comprises a packetized elementary stream and said 
apparatus further comprises: 

a transport stream encoder, coupled to said multiplexer 
output, for transport encoding said output bitstream to 30 
produce a transport encoded bitstream. 

18. The apparatus of claim 17 wherein said transport 
stream encoder receives a second bitstream from a second 
bitstream source, and said transport encoded bitstream com- 



prises at least said output bitstream from said multiplexer 
and said second bitstream. 

19. The apparatus of claim 14 further comprising: 

a frame byte counter, coupled to said logic unit, for 
counting a number of bytes stored in a decoder buffer; 

said logic unit, in response to said number of bytes stored 
in said decoder buffer exceeding a predetermined maxi- 
mum amount, causing said buffer control signal to 
indicate that said desired flow is decreased; and 

said logic unit, in response to said number of bytes stored 
in said decoder buffer being below a predetermined 
minimum amount, causing said buffer control signal to 
indicate that said desired flow is increased. 

20. The apparatus of claim 15 further comprising: 

a frame byte counter, coupled to said logic unit, for 
counting a number of bytes stored in a decoder buffer; 

said logic unit, in response to said number of bytes stored 
in said decoder buffer exceeding a predetermined maxi- 
mum amount, causing said buffer control signal to 
indicate that said desired flow is decreased; and 

said logic unit, in response to said number of bytes stored 
in said decoder buffer being below a predetermined 
minimum amount, causing said buffer control signal to 
indicate that said desired flow is increased. 

21. The apparatus of claim 13 wherein said output bit- 
stream comprises a packetized elementary stream and said 
apparatus further comprises: 

a transport stream encoder, coupled to said multiplexer 
output, for transport encoding said output bitstream to 
produce a transport encoded bitstream. 
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